Model Violation Analysis

The plots and tables on this tab summarize how well the GST estimate actually fits the data. Although they may be less familiar to you than, say, process fidelity, these analyses are essential to understanding the GST estimate, and how much to trust it! Real qubit systems often display behavior that isn't consistent with modeling each gate as a stationary CPTP map. When pyGSTi tries to fit such data to its model (a single gateset), this "non-Markovianity" manifests as model violation. This tab provides several views of the model violation observed for this fit, ranging from the coarse-grained (total violation) to hyper-detailed (per-circuit violation). More observed model violation ⇒ the error metrics on other tabs should be trusted less.

Model violation for each individual circuit

This tab presents the second of two statistical tests available to detect that a dataset violates pyGSTi's Markovian gateset model. The first is based on the aggregate loglikelihood score presented elsewhere. This tab shows the individual loglikelihood scores for each circuit in the dataset. Each circuit is represented by a colored box, whose color indicates how badly this estimate (model) failed to predict the observed outcome frequencies of that circuit. Light gray indicates consistency with the model, dark gray indicates possible -- but statistically insignificant -- inconsistency, and red squares indicate circuits whose outcomes are inconsistent with the model at the family-wise 95 percent confidence level. In other words, if data are generated by a Markovian gateset, then with 95%% probability every box will be gray. Even a single red square thus represents a clear detection of model violation. When many squares are red, their pattern can provide useful diagnostic clues to what kind of non-Markovian noise is present.

Gauge Invariant Error Metrics

GST can estimate gates up to an overall gauge. PyGSTi tries to find a good gauge in which to report process matrices and gauge-variant metrics like fidelity -- but sometimes this goes wrong. The most reliable error metrics and gate properties are gauge-invariant ones, and these are listed on this tab.

Gauge Invariant Error Metrics applied to germ sequences

All of the per-gate gauge-invariant metrics of the previous tab are functions of each gate's spectrum and do not account for how the gate relates to other gates. In an attempt to extract some of that information in a gauge-invariant way, this tab looks at the spectra of the germ-sequences. Each germ amplifies (i.e. has eigenvalues which correspond to) certain directions in gate-set space. Some of these directions describe how the single gates relate to one another, and, if an amplificationally complete set of germs was used, every direction is amplified by at least one germ. This implies that the (gauge-invariant) spectra of the germs should constitute a full description of the gate set. This tab compares each germ-spectrum to the spectrum of that germ if it were generated using the set eigenspace-projected gates obtained by placing each gate's GST-estimated eigenvalues within eigenbasis of the ideal target gate.

Gauge Variant Error Metrics

This tab provides a variety of common (and uncommon) error metrics derived from the estimated gate set. All of these quanties are gauge-dependent, which means two things. First, they aren't directly physically measurable, so they can't map directly to observable error rates. Second, they are only as reliable (as diagnostics) inasmuch as pyGSTi is able to pick a sensible gauge (reference frame) in which to report gates. PyGSTi does this by first finding an estimate based on the data (and ignoring gauge entirely), then varying over all possible representations of those gates (gauges) to minimize a measure of the gates' implausibility (distance from the targets, combined with violation of positivity). This measure has parameters -- e.g. the weights placed on different gates -- and reports often include multiple "gauge optimizations". A dropdown menu in the sidebar allows switching between these options, and the parameters used for the currently-shown estimate are shown below it.

Raw Estimates

This tab shows the raw GST estimates — the density matrices, POVM effects, and process matrices that comprise the estimated gateset. These are explicitly gauge-dependent, and most reports include multiple gauge optimizations that correspond to slightly different (but physically equivalent and indistinguishable) representations of the operations. Usually, these raw estimates are less useful than the derived properties and decompositions shown elsewhere, but sometimes it's useful to see them. Furthermore, it's possible (at least in some versions of the report) to download the raw gateset in machine-readable form, in order to do calculations and simulations with it.

Gate Decompositions

This tab presents several decompositions of the process matrices shown on the Raw Estimates tab. These are derived properties of the process matrices that aren't directly interpretable as error metrics, but help understand both the overall observed/estimated behavior of the gates and how they differ from the targets. Although the tables on this page do not compare the estimated gates' properties directly to those of the ideal targets, many reports include an analysis of the target gates as well, which can be accessed (and compared directly to the GST estimates) through the Estimates dropdown menu on the sidebar. Also, since these properties are all at least mildly gauge-dependent, it may be useful to examine different gauge-optimization choices using that dropdown menu on the sidebar.

Gate Error Generators

This tab presents the error generators for each of the estimated gates. Although these are not especially well-known in the literature, they are (in the pyGSTi authors' opinion) the most useful detailed diagnostic for gate errors. The error generator \mathbb{L} for a noisy gate G with ideal target G_0 is defined by writing G = e^{\mathbb{L}} G_0 . It can be thought of, more or less, as a Lindbladian superoperator that generates the error in the gate — with two caveats. First, it is not necessarily of strict Lindblad form, because the GST-estimated gates may not be CP, and because even if they are, not every CP map is "divisible" (and nondivisible maps are not generated by Lindblad evolution). Second, the generator reported here is a post-gate generator, so it answers the question "If all the noise occurred after the ideal gate, what Lindbladian would generate it?" Finally: the error generators are very definitely gauge-dependent, so caveat emptor (cross-validating any inferences drawn from these generators with some sort of gauge-invariant diagnostic is highly recommended).

Input Summary

This tab presents a grab bag of potentially-useful information about the target gate set and the data set(s).

System and pyGSTi parameters

This tab contains a raw dump of system information and various pyGSTi parameters. Its purpose is to stamp this report with parameters that indicate how exactly GST was run to create it, as well as to record the software environment in within which the report creation was run. However, if the core GST computation was done on a different computer, then the software information contained here will be less useful (it describes the environment in which the report was generated, not the one in which the estimate was generated).

How to use this report

Welcome to a pyGSTi report! This HTML document provides an interactive view onto a lot of information about one or more gateset tomography (GST) experiments that were analyzed using the pyGSTi software. This Help tab should help you use this report's interactive features, figure out where to find things, and access the context-sensitive help features embedded throughout the report.

Background

PyGSTi's primary purpose is to ingest GST datasets and find a good estimate of the gateset that generated that data. But pyGSTi can also produce a lot of detailed analyses and visualizations of the estimates that it generates. That's what you'll find in this report. You can inspect the raw estimated gateset too, but if you want to do your own calculations with it — e.g., to simulate experiments or calculate interesting quantities that the pyGSTi authors haven't thought of — then the best way to do that is by using pyGSTi itself in a Jupyter or iPython notebook.

This report's structure

PyGSTi uses HTML for reporting because it's pretty portable and supports interactivity. Interactivity makes it possible to embed a lot of different perspectives and analyses, without overwhelming you with too much information at once or requiring you to scroll through long static (e.g. PDF) documents. In this report, actual content is displayed in the main panel, while navigation tools that let you decide what content to examine are collected in a sidebar on the left. The tables and figures in the main panel are also somewhat interactive (see “Interacting with content” below). Most of them dynamically change what they show depending on the positions of one or more switches - dropdown-menus, button-sets, or sliders. Most switches are located on the sidebar, but some appear in the main panel beneath the figure or table that they control. Note that some content is generated on the fly, so there may be a lag the very first time you view a given tab, while your browser makes plots or renders math formulas.

Choosing what to see via the sidebar

The sidebar provides two distinct ways to control what you see. First, the data analysis is broken up into different tabs that are listed on the sidebar. Use the sidebar to hop between them. Each tab collects a particular category of analysis, presented as tables and figures. Second, if the report contains multiple analyses, the sidebar displays switches that allow you to pick which analysis to display in the main panel. A report can contain: (1) multiple distinct datasets; (2) the results of different estimators (each associated with a given dataset); and (3) multiple distinct but equivalent representations (gauges) for each estimate. You can choose between these options using the drop down menus labeled Dataset and Estimate, and the Gauge-Opt. button group at the bottom of the sidebar. If a switch is missing, the report doesn't contain multiple options of that type. You can hide the sidebar (or make is sticky) by clicking the ⊛ symbol in its upper right corner.

Due to the large amount of data a report can contain, there are options that allow one to omit some of the (usually larger or more data-dense) figures from the report at its creation time. This will result in the report having fewer tabs in the sidebar and/or the word OMITTED being displayed instead of a figure. To see missing tabs or omitted figures, you should ask whoever generated this report to re-generate it with a lower brevity setting.

Interacting with content

Each tab presents a (hopefully manageable) amount of content arranged into objects — figures and tables — that have some interactive properties. Here are some general tips and rules.

• Resizing: Each object can be resized by dragging its lower right corner.
• Zooming: You can zoom in on plots' axes by clicking and dragging within the figure. Double-clicking, or tapping the house icon that pops up in the upper right when the pointer is over a figure, will reset the axes.
• Exporting: Plots can be saved in PNG format by tapping the camera icon, which only appears when the pointer is over a plot). (User beware: this is a builtin feature of the Plotly library, and we find that sometimes the saved PNG does not look like the original plot, especially in aspect ratio.) Depending on what options were selected when the report was constructed, there may also be links to download plots and tables as PDF, Python pickle files, and/or LaTeX (tables only). Plots will have additional download icons next to the camera icon, and tables will have PDF, PKL, and/or TEX links beneath them. PKL files downloaded from plots contain a pickled dictionary of the plotted data, while those downloaded from tables contain a pickled Pandas DataFrame object.
• Full descriptive captions for figures and tables can be revealed by clicking on the object's Figure XX or Table XX title.
• Hovering over a colored bar or box will often reveal a tool tip showing the exact number that bar or box represents. Hovering over table headings will bring up descriptive text. In the plots that show per-sequence model violation, hovering over any data point will bring up fairly extensive details about the datum that generated it.
• Clickable pigs: If you see the pyGSTi logo, it's a placeholder for a plot that wasn't automatically generated (usually because this might take a little while); click on the logo to generate the plot.

Quick guide to Datasets, Estimates, and Gauge Optimizations

The single biggest change from the old PDF report format is the ability to report simultaneously on — and compare — multiple gateset estimates. This report may contain an unlimited number of distinct gatesets. Although they're often all derived from the same dataset, they may not be. For example, a single report could contain:

1. datasets from separate GST experiments on each qubit on a chip;
2. datasets from two identical GST experiments done at different times;
3. datasets from GST experiments on the same system, but with gate implementations;
4. a 2-qubit GST dataset together with 1-qubit GST datasets extracted from it;
5. many other possibilities.

The sidebar's Dataset dropdown menu (if present) lets you select a dataset. Then, the Estimate dropdown (if present) will list all the different estimates that are available for any of the datasets in the report. If you choose an estimate that was not generated for the current dataset (but is listed because it was generated for another dataset), unavailable plots and figures will turn into big "N/A" labels. Different estimates are usually generated by different statistical estimators. PyGSTi usually provides maximum likelihood estimates (although others are certainly possible!), but it can do MLE with no constraints at all (Full), or constrained to trace-preserving maps (TP), or constrained to completely positive maps (CPTP). Some reports include separate *.robust variations of those estimators in which poorly-fit data were deprecated, and some reports include the target gates themselves (Target) as an option.

Finally, in order to analyze each estimated gateset, pyGSTi has to choose a representation — a gauge — for it. Since pyGSTi does this gauge-fixing internally by defining an objective function and then finding the gauge that optimizes it, the choice-of-representation is referred to as Gauge Optimization. A description of the objective function used to generate this gateset appears at the bottom of the sidebar. If the report contains multiple gauge optimizations, you can choose between them by pushing the appropriate button. Choosing a good gauge is hard, and neither pyGSTi nor its users always get it right. If something reported in the Summary or Gauge-dependent error metrics tabs looks startling or implausible, it might be due to a poorly chosen gauge. Try a different gauge-optimization choice, or cross-validate whatever you see against things from the Gauge-invariant error metrics tabs.

What to look at first (and second)

If you're new to pyGSTi reports, this is probably all a bit overwhelming. Don't panic! There's a lot of context-sensitive help, and the pyGSTi authors are always happy to answer questions like What does this mean?, or Why would I care about this? But since these reports do contain a lot of stuff, here's a get-started-quickly guide.

Start with the Summary tab. It will tell you two critical things. First, in Table 3: how close do the gates seem to be to the ideal targets that were specified? Are you doing pretty well, or all messed up for some reason? If whoever did the analysis chose to produce error bars (this can take a lot of computation), they'll appear in this table. Second, the model violation plots will tell you whether the system seems to be stable and Markovian, or not (in which case, take the metrics in Table 3 with a grain of salt). Don't forget to flip through the available datasets, estimates, and gauge-optimization choices — or at least the ones you care about — while looking at this tab.

Next, you probably want to know one of two things. Either the GST model was NOT badly violated, in which case you want to know more about the errors in the gates.. or it WAS badly violated, in which case you want to know more about what's going wrong.

If you want to know more about the errors, start with Gauge Dependent Error Metrics: Overview, then take a quick look at Raw Estimates to confirm that the process matrices look about right, and then spend some quality time with Gate Decompositions and Gate Error Generators if you want to really understand what GST thinks that each individual gate is doing. If at any point you see something weird that might be due to a bad choice of gauge (examples include wildly nonpositive SPAM operations, implausibly high diamond norm errors in gates that you strongly believe to be better than that, and significantly negative elements in the Pauli stochastic part of the error generators), try a different gauge optimization. Or, go to the Gauge-invariant error metrics tabs and try to figure out whether there's something here that confirms or denies whatever effect is bugging you (e.g., if an estimated gate has an eigenvalue bigger than 1, that will produce crazy negative probabilities, and it's not a gauge problem).

On the other hand, if there's a lot of model violation, then you probably want to understand why. The information on the error metrics tabs may or may not be useful — you should be much more cautious about drawing conclusions from them if there's a lot of model violation. To dig into model violation (aka “non-Markovianity”), start with the Model Violation: Overview tab, which will give you the numbers behind the bar chart from the Summary tab, and also show you whether longer circuits violate the GST model more. (If they do, it's probably non-Markovianity in the gates. If they don't, it's probably something nastier like bistability, slow drift in SPAM operations, or corrupted data). Next, take a deep breath and dig into the “Per-sequence detail” tab, which will show you the (quantitative) inconsistency of every individual circuit with this estimate. Interpreting these charts is a bit of an art, but clusters of red boxes usually indicate that something related to the location of the cluster was suffering drift and/or non-Markovianity.